Fast Solutions to Projective Monotone Linear Complementarity Problems

We present a new interior-point potential-reduction algorithm for solving monotone linear complementarity problems (LCPs) that have a particular special structure: their matrix M ∈ Rn×n can be decomposed as M = ΦU +Π0, where the rank of Φ is k < n, and Π0 denotes Euclidean projection onto the nullspace of Φ⊤. We call such LCPs projective. Our algorithm solves a monotone projective LCP to relative accuracy ǫ in O( √ n ln(1/ǫ)) iterations, with each iteration requiring O(nk) flops. This complexity compares favorably with interior-point algorithms for general monotone LCPs: these algorithms also require O( √ n ln(1/ǫ)) iterations, but each iteration needs to solve an n × n system of linear equations, a much higher cost than our algorithm when k ≪ n. Our algorithm works even though the solution to a projective LCP is not restricted to lie in any low-rank subspace. 1 Linear complementarity problems The LCP for a matrix M ∈ Rn×n and a vector q ∈ R is to find vectors x, y ∈ R with x ≥ 0 y ≥ 0 y = Mx+ q x⊤y = 0 (1) We say that vectors x, y are feasible if they satisfy the first three conditions of (1) (i.e., leaving off complementarity), and we call them a solution if they satisfy all four conditions. The complementarity gap x⊤y is nonnegative for any feasible point (x, y), and measures how close a feasible point is to being a solution. (See [1] for an overview of LCPs.) This work was first presented at the 2010 NIPS workshop “Learning and Planning from Batch Time Series Data.” If M is positive semidefinite (but not necessarily symmetric), the LCP ismonotone, and there exist interiorpoint algorithms that solve it to relative accuracy ǫ in O( √ n ln(1/ǫ)) Newton-like iterations. In each iteration, the main work is to solve an n × n system of linear equations. Suppose the matrix M can be decomposed as M = ΦU + Π0, where Φ ∈ Rn×k and U ∈ Rk×n have rank k < n, and Π0 projects onto the nullspace of Φ ⊤ (that is, Π0 = I −ΦΦ†, where † denotes the Moore-Penrose pseudoinverse). In this case we call (M, q) a projective LCP of rank k. Our new algorithm solves a projective LCP in O( √ n ln(1/ǫ)) iterations, the same as for the general monotone case, but with each iteration requiring only O(nk) flops. This result is an analog of the situation for linear equations: a rank-k factored system of linear equations can be solved in O(nk) flops, while it is believed that a general n × n system of equations requires Ω(n) flops for some constant η > 0. However, unlike the situation for linear equations, in a projective LCP we can’t a priori restrict either x or y to a low-rank subspace of R; so, it is perhaps surprising that the analogous complexity result still holds. (The inequality constraints x ≥ 0 and y ≥ 0 are the source of this difficulty: the intersection of x ≥ 0 or y ≥ 0 with a rank-k subspace can be quite restrictive.) 2 Potential reduction We say that x and y are strictly feasible if they satisfy x > 0, y > 0, and y = Mx + q. We will assume that we know a strictly feasible initial point (x0, y0) for our LCP. (If we do not, it is possible to construct one, as mentioned in [2].)